Abstract
Human activity recognition (HAR) based on mobile sensors plays an important role in ubiquitous computing. However, the rise of data regulatory constraints precludes collecting private and labeled signal data from personal devices at scale. Thanks to the growth of computational power on mobile devices, federated learning has emerged as a decentralized alternative solution to model training, which iteratively aggregates locally updated models into a shared global model, therefore being able to leverage decentralized, private data without central collection. However, the effectiveness of federated learning for HAR is affected by the fact that each user has different activity types and even a different signal distribution for the same activity type. Furthermore, it is uncertain if a single global model trained can generalize well to individual users or new users with heterogeneous data. In this paper, we propose Meta-HAR, a federated representation learning framework, in which a signal embedding network is meta-learned in a federated manner, while the learned signal representations are further fed into a personalized classification network at each user for activity prediction. In order to boost the representation ability of the embedding network, we treat the HAR problem at each user as a different task and train the shared embedding network through a Model-Agnostic Meta-learning framework, such that the embedding network can generalize to any individual user. Personalization is further achieved on top of the robustly learned representations in an adaptation procedure. We conducted extensive experiments based on two publicly available HAR datasets as well as a newly created HAR dataset. Results verify that Meta-HAR is effective at maintaining high test accuracies for individual users, including new users, and significantly outperforms several baselines, including Federated Averaging, Reptile and even centralized learning in certain cases. Our collected dataset will be open-sourced to facilitate future development in the field of sensor-based human activity recognition.
Introduction
Human Activity Recognition (HAR)—utilizing smartphone sensors like accelerometers and gyroscopes to detect activities such as walking, biking, or sitting—is a cornerstone of ubiquitous computing. However, its deployment faces two significant hurdles: privacy and heterogeneity. Data regulatory constraints like GDPR make centralizing raw sensor data difficult. Simultaneously, standard Federated Learning (FL) assumes Independent and Identically Distributed (IID) data, which contradicts the reality of HAR. Users exhibit label heterogeneity (different activity sets) and signal heterogeneity (different movement patterns for the same activity due to hardware or physiological differences). Conventional FL methods like Federated Averaging (FedAvg) often struggle, seeing accuracy drops of up to 20% compared to centralized training because they attempt to force a single global classifier onto diverse user profiles.
Methodology
Meta-HAR addresses these challenges by shifting the focus from learning a global classifier to learning a global representation. The framework decouples the model into two components: a Signal Embedding Network and a Personalized Classification Network.
The Signal Embedding Network uses CNNs for spatial features and RNNs for temporal dependencies. It is trained using a federated version of Reptile, a first-order meta-learning algorithm, to ensure the model is "easy to adapt" to any user's specific task. A critical innovation is the use of Pairwise Similarity Loss. Instead of standard cross-entropy, the embedding network is trained to recognize if two signal segments belong to the same activity. This Siamese network approach allows the model to cluster similar movements in a latent space, making representations robust to varying movement styles and bypassing the need for a fixed global label map. When a new user joins, they pull the global embedding network and perform a two-stage adaptation: refining embeddings with their own data and then training a local classification head.
Experimental Results
The researchers evaluated Meta-HAR on three datasets: HHAR (Heterogeneous HAR), USC-HAD, and a new, noisy "Collected Dataset" featuring 48 users with unconstrained behavior.
Key findings include:
- Superior Accuracy: Meta-HAR achieved test accuracies of 92.5% to 98.4% on public datasets, significantly outperforming FedAvg, which struggled in the 69-84% range.
- Adaptation Capability: In "meta-test" scenarios with unseen users, Meta-HAR outperformed models trained on centrally collected data, demonstrating its superior ability to generalize and adapt.
- Robustness to Non-IID Data: On highly heterogeneous datasets, Meta-HAR maintained high performance (~95% for meta-train users) while baselines like FedReptile saw significant performance degradation.
Conclusion
Meta-HAR provides a robust framework for personalized human activity recognition in a privacy-preserving federated environment. By prioritizing representation learning over universal classification, it effectively handles the inherent signal and label heterogeneity of mobile sensor data. While it requires a small amount of local labeled data for personalization, its ability to outperform centralized models in adaptation scenarios marks a significant step forward for decentralized HAR. Future work will explore reducing the labeling burden and optimizing communication for low-power IoT devices.
Summary updated