Opportunistic Federated Learning: An Exploration of Egocentric Collaboration for Pervasive Computing Applications

Sangsu Lee, Xi Zheng, Jie Hua, Haris Vikalo, Christine Julien

Abstract

Opportunistic Federated Learning (OFL) is a decentralized framework designed for pervasive computing, where mobile devices learn personalized models through fleeting peer-to-peer encounters. Unlike traditional Federated Learning, OFL eliminates the need for a central coordinator and addresses data heterogeneity through an egocentric approach. By introducing "opportunistic momentum," the framework selectively incorporates gradients from neighbors, enhancing local model accuracy and resilience to diverse data distributions.

Introduction

Standard Federated Learning (FL) typically relies on a central server to aggregate data from thousands of devices to build a single, global model. However, in pervasive computing—such as smartphone text prediction or wearable gait recognition—users have distinct needs and encounter vastly different data. The authors identify three major gaps in current FL: the requirement for a persistent central coordinator, the assumption of model homogeneity (ignoring personalization), and the failure to account for the "opportunistic" nature of mobile connectivity, where devices may only be in range for seconds.

The OFL Framework

The researchers propose an egocentric approach where each device's primary objective is to improve its own local, personalized model rather than contributing to a global goal.

Overview of the Opportunistic Federated Learning (OFL) architecture, showing the transition from a bootstrap model to decentralized, peer-to-peer personalization.

Key components of the OFL system include:

  • The Encounter Filter: Devices exchange data label distributions to calculate a similarity score. Encounters with neighbors whose data is too dissimilar are ignored to prevent model "pollution."
  • Feasibility Check: The system uses mobility patterns to predict encounter duration, ensuring enough time for a training round.
  • Opportunistic Momentum: Instead of simple averaging (Greedy Aggregation), devices maintain a table of gradients from recent diverse encounters. A windowed average provides stability and prevents overfitting to the most recent neighbor.
  • Dynamic Decay: The learning rate decays based on the L2 distance from the initial "bootstrap" weights to keep the model from drifting too far from a reliable state.

Experimental Evaluation

The framework was evaluated using MNIST and CIFAR-10 datasets, modified to simulate extreme data skew where each device only sees a small subset of labels. Human movement was simulated using the Levy walk mobility model in a 9-region grid.

Key Results

The results demonstrate that collaboration without consensus is highly effective. Devices benefit from each other without needing to agree on a single global model.

Performance comparison on MNIST and CIFAR-10 datasets, demonstrating the stability and superior accuracy of the opportunistic-momentum approach compared to greedy and local-only methods.
  • Resilience to Shift: In experiments with changing neighbor types, opportunistic-momentum maintained high accuracy (approx. 93% for MNIST), while "greedy" approaches suffered from overfitting to unbalanced data.
  • Personalization Benefits: OFL-trained models consistently outperformed "local-only" models, proving that even fleeting interactions provide significant signals for sharpening personalized models.
  • Feasibility: Benchmarks on a Raspberry Pi 4 showed that a full training round takes about 19 seconds over WiFi-Direct, making it practical for real-world encounters like waiting in a coffee line.

Conclusion and Future Work

OFL presents a viable path for decentralized, personalized machine learning in mobile environments. However, real-world deployment faces challenges such as the need for perfect encounter duration prediction and the development of incentive structures for peer participation. Future work will also explore adapting to dynamic user goals as environments change.

Summary updated

Abstract

Pervasive computing applications commonly involve user's personal smartphones collecting data to influence application behavior. Applications are often backed by models that learn from the user's experiences to provide personalized and responsive behavior. While models are often pre-trained on massive datasets, federated learning has gained attention for its ability to train globally shared models on users' private data without requiring the users to share their data directly. However, federated learning requires devices to collaborate via a central server, under the assumption that all users desire to learn the same model. We define a new approach, opportunistic federated learning, in which individual devices belonging to different users seek to learn robust models that are personalized to their user's own experiences. However, instead of learning in isolation, these models opportunistically incorporate the learned experiences of other devices they encounter opportunistically. In this paper, we explore the feasibility and limits of such an approach, culminating in a framework that supports encounter-based pairwise collaborative learning. The use of our opportunistic encounter-based learning amplifies the performance of personalized learning while resisting overfitting to encountered data.
BibTeX
@inproceedings{lee2021opportunistic,
  author = {Sangsu Lee and Xi Zheng and Jie Hua and Haris Vikalo and Christine Julien},
  title = {{Opportunistic Federated Learning: An Exploration of Egocentric Collaboration for Pervasive Computing Applications}},
  booktitle = {PerCom},
  year = {2021},
  url = {https://openreview.net/forum?id=oUFASwltp0},
}
Figure