Abstract
Federated Learning (FL) aims to train robust global models across decentralized devices while maintaining data privacy. While significant research has focused on inter-client heterogeneity (the variance of data distributions between users), the challenge of intra-client heterogeneity—where a single user's data contains multiple, diverse sub-domains—remains largely unaddressed. This paper introduces FedIns, a novel framework designed for Federated Instance-adaptive inference. By utilizing a pool of lightweight "Scale and Shift" (SSF) parameters and a dynamic routing mechanism, FedIns allows models to adapt to individual data instances on the fly. Experimental results demonstrate that FedIns achieves up to a 7.77% accuracy improvement over traditional FL methods while reducing communication bandwidth requirements by 85%.
1. Introduction
In standard Federated Learning (FL) paradigms, the primary objective is to mitigate the performance degradation caused by non-I.I.D. (independent and identically distributed) data across different clients. However, real-world data collection often introduces "intra-client heterogeneity." For instance, a single medical imaging device may process scans from various manufacturers, or a mobile photo gallery may contain a mixture of sketches, photographs, and digital art.
Traditional FL algorithms typically attempt to learn a single global model or a single personalized model per client. These approaches often fail when a client's local data is highly diverse, as the model essentially tries to "average" across conflicting data distributions. To solve this, we propose FedIns, which shifts the focus from client-level adaptation to instance-level adaptation.
2. The FedIns Framework
FedIns introduces a parameter-efficient approach to handle complex, mixed-style data within individual devices. The framework is built upon two core components: Parameter-Efficient Fine-Tuning (PEFT) and an Instance-Adaptive Inference mechanism.
2.1 Parameter-Efficient Fine-Tuning via SSF
Rather than updating the entire weight matrix of a deep neural network—which is computationally expensive and communication-heavy—FedIns employs Scale and Shift deep Features (SSF). SSF modulates the features of a pre-trained backbone model using a minimal set of parameters. This allows the model to be "nudged" toward specific data distributions without the need for full-model fine-tuning.
2.2 The SSF Pool and Dynamic Routing
Instead of maintaining a single set of SSF parameters, each client in FedIns manages an SSF Pool. When the model encounters a new input instance, it performs the following steps:
- Query Generation: The pre-trained backbone generates a unique query for the specific input image.
- Key Matching: This query is compared against a set of "keys" associated with the SSF subsets in the pool.
- Instance-Adaptive Aggregation: The model selects and aggregates the best-matched SSF subsets to create a custom, instance-specific model for that single inference task.
This dynamic routing ensures that a "sketch" image is processed by parameters optimized for sketches, while a "photograph" in the same dataset is processed by parameters optimized for realism.
3. Experimental Evaluation
3.1 Datasets and Setup
The authors evaluated FedIns against three major benchmarks modified to simulate both inter- and intra-client diversity:
- CIFAR-100 & Tiny-ImageNet: Used to evaluate performance under "Label Shift," where class distributions vary across clients.
- DomainNet: A large-scale dataset featuring six distinct domains (Clipart, Infograph, Painting, Quickdraw, Real, and Sketch), used to simulate "Feature Shift."
3.2 Performance and Efficiency
FedIns was compared against state-of-the-art FL algorithms, including FedAvg, FedProx, and FedBN. The results highlight three key advantages:
- Superior Accuracy: On the Tiny-ImageNet benchmark, FedIns achieved a 7.77% accuracy boost over the baseline FedAvg. It consistently outperformed competitors across all tested levels of data heterogeneity.
- Communication Efficiency: By only transmitting the lightweight SSF parameters (approximately 5.35 million parameters) rather than the full model (85.8 million parameters), FedIns reduces communication costs by over 85%.
- Robustness: As data became increasingly heterogeneous, the performance of traditional algorithms declined sharply. In contrast, FedIns maintained stable performance, proving its resilience to "messy" real-world data.
4. Discussion and Limitations
While FedIns represents a significant step forward, certain considerations remain for deployment. The performance of the system is sensitive to hyperparameters, specifically the size of the SSF pool ($M$) and the number of subsets selected for aggregation ($C$). Additionally, while communication costs are low, the instance-specific matching process introduces a small amount of computational overhead during inference, which may be a factor for ultra-low-power edge devices.
5. Conclusion
FedIns addresses the neglected challenge of intra-client heterogeneity in Federated Learning. By moving toward instance-adaptive inference, the framework provides a scalable, efficient, and highly accurate solution for decentralized learning on diverse datasets. Future work will focus on extending this instance-adaptive logic to other domains, such as Natural Language Processing and time-series analysis.
Summary updated