Federated Learning with Only Positive Labels by Exploring Label Correlations

Xuming An, Dui Wang, Li Shen, Yong Luo, Han Hu, Bo Du, Yonggang Wen, Dacheng Tao

Abstract

Federated Learning (FL) aims to train models collaboratively while preserving data privacy. However, multi-label classification in FL faces a critical failure mode when clients only possess positive data for a single class—a scenario common in biometric verification and personalized user models. This "only positive labels" setting leads to "embedding collapse," where all class representations converge to a single point, rendering the model non-discriminative. We introduce FedALC (Federated Averaging by exploring Label Correlations), a framework that leverages label correlations to regularize the embedding space. By using privacy-preserving hashing to detect co-occurrences and applying a correlation-aware geometric regularizer, FedALC prevents collapse and improves discriminative power. Our variant, FedALC-fixed, further optimizes communication by using static embeddings. Extensive evaluations on image and extreme multi-label text datasets demonstrate that FedALC significantly outperforms existing baselines.

Introduction: The Challenge of Single-Positive Label FL

Standard FL assumes data diversity across clients. In reality, a client (e.g., a mobile phone) might only contain images of its owner ("Person" label) and no images of other people or objects. In a multi-label setting, training on only positive instances causes the model to pull instance embeddings toward the target class without any negative pressure to push them away from others. This leads to a useless model where all outputs are identical. Previous solutions like FedAwS attempted to force class embeddings apart uniformly, but this ignores the inherent semantic relationships between labels (e.g., "Car" and "Truck" should naturally be closer in embedding space than "Car" and "Cloud").

The FedALC Framework

FedALC addresses this by incorporating label correlations into the server-side regularization. The architecture involves a multi-step process between the server and edge clients to ensure privacy while building a semantically meaningful model.

Overview of the FedALC architecture showing the interaction between the server and edge clients.

The process involves:

  1. Privacy-Preserving Hashing: Clients compute irreversible hashes (e.g., SHA-256) of their data. If two clients with different labels produce the same hash, the server infers a label co-occurrence without ever seeing the raw private data.
  2. Correlation-Aware Regularization: The server calculates weights based on these co-occurrences. It then applies a geometric regularizer that pushes class embeddings apart, with the force modulated by the correlation weight—highly correlated labels are pushed less aggressively.

Optimization and Efficiency

FedALC uses positive class embeddings as "surrogates" for private instance embeddings on the server side. This allows the server to perform the necessary contrastive "pushing" that clients cannot do locally due to their lack of negative samples.

Optimization process in the class embedding space, illustrating how positive and negative embeddings are managed between the server and client.

To handle "Extreme Multi-label" (XML) scenarios with hundreds of thousands of classes, we propose FedALC-fixed. In this variant, the server pre-calculates and fixes the class embedding matrix based on label correlations before the FL process begins. This eliminates the need for frequent, bandwidth-heavy updates of the embedding layer, significantly reducing communication overhead and privacy leakage risks.

Experimental Results

FedALC was tested on diverse image datasets (COCO, VOC, FLAIR) and XML text datasets (Amazon670K, WikiLSHTC).

  • Image Tasks: On VOC 2012, FedALC achieved an 8.17% relative improvement in Mean Average Precision (MAP) over the state-of-the-art FedAwS.
  • Text Tasks: In the Bibtex dataset, it saw a 19.3% boost in Precision@1 (P@1).
  • Communication Efficiency: FedALC-fixed proved particularly effective in high-dimensional label spaces, reducing communication costs while maintaining or exceeding the performance of dynamic embedding methods.

Conclusion

FedALC provides a robust solution for multi-label FL under extreme data imbalance. By acknowledging that labels are semantically linked rather than independent entities, it creates a more realistic and effective embedding space. Future work will focus on refining the surrogate embedding strategy to further minimize information loss during server-side optimization.

Summary updated

Abstract

Federated learning aims to collaboratively learn a model by using the data from multiple users under privacy constraints. In this paper, we study the multi-label classification problem under the federated learning setting, where trivial solution and extremely poor performance may be obtained, especially when only positive data w.r.t. a single class label are provided for each client. This issue can be addressed by adding a specially designed regularizer on the server-side. Although effective sometimes, the label correlations are simply ignored and thus sub-optimal performance may be obtained. Besides, it is expensive and unsafe to exchange user's private embeddings between server and clients frequently, especially when training model in the contrastive way. To remedy these drawbacks, we propose a novel and generic method termed Federated Averaging by exploring Label Correlations (FedALC). Specifically, FedALC estimates the label correlations in the class embedding learning for different label pairs and utilizes it to improve the model training. To further improve the safety and also reduce the communication overhead, we propose a variant to learn fixed class embedding for each client, so that the server and clients only need to exchange class embeddings once. Extensive experiments on multiple popular datasets demonstrate that our FedALC can significantly outperform existing counterparts.
BibTeX
@article{an2025federated,
  author = {Xuming An and Dui Wang and Li Shen and Yong Luo and Han Hu and Bo Du and Yonggang Wen and Dacheng Tao},
  title = {{Federated Learning with Only Positive Labels by Exploring Label Correlations}},
  journal = {IEEE Trans. Neural Networks Learn. Syst.},
  year = {2025},
  url = {https://openreview.net/forum?id=Vt1vWOfA2T},
}
Figure