FedKLS: Federated KL-Driven Low-rank SVD Adaptation in Non-IID Data Distributions
Its authors withdrew this paper, so there is no full text to read or summarize.
Lists
Only you see this
Abstract
Federated learning faces two key challenges: handling non-IID client distributions and reducing communication costs in adapting large models. To address these issues, we propose FedKLS, a framework that combines KL-divergence-based personalization with low-rank SVD-based adaptations. FedKLS chooses spectral components in a dynamical manner by mapping the heterogeneity of client distribution to the singular value spectrum, then builds specialized LoRA-style adapters, which allow aggregation at scale and client-specific specialization. Extensive experiments on 20NewsGroup and Banking77 with DistilBERT and Qwen backbones show that FedKLS achieves competitive performance compared to state-of-the-art parameter-efficient fine-tuning baselines, including LoRA, PiSSA, MiLoRA, and full fine-tuning. In highly non-IID settings ($\alpha = 0.01$), FedKLS improves F1-score by up to 11--12\% and reduces total communication cost by about 3x over PiSSA, achieving the best trade-off between personalization and scalability. These results demonstrate the effectiveness of KL-guided spectral adaptation in federated fine-tuning of large models. Our implementation is available at: https://anonymous.4open.science/r/FedKLS.
BibTeX
@misc{quan2026fedkls,
author = {Pham Khanh Quan and Khoa Nguyen and Leo Yu Zhang and Madhusanka Liyanage and Taehong Kim},
title = {{FedKLS: Federated KL-Driven Low-rank SVD Adaptation in Non-IID Data Distributions}},
year = {2026},
url = {https://openreview.net/forum?id=gxKvAhqhmT},
howpublished = {OpenReview},
note = {Submitted to ICLR 2026},
}