TL;DR
This paper introduces DivAR, a federated meta-learning framework for human activity recognition (HAR) that tackles the challenges of data privacy and individual diversity. By clustering users based on their characteristics and employing attention mechanisms within a federated learning setup, DivAR achieves robust and personalized activity recognition, even for new users.
What Problem Does This Paper Address?
Human Activity Recognition (HAR) using smartphone sensors holds immense potential for applications like smart homes and health monitoring. However, two major hurdles impede its widespread adoption:
- Data Privacy: Deep learning models for HAR require vast amounts of sensor data, which are sensitive and cannot be easily shared with a central server due to privacy concerns.
- Individual Diversity: People exhibit diverse behavioral patterns and lifestyles, leading to significant variations in sensor data even for the same activity. This "heterogeneity" makes it difficult for a single model to perform well across all users, especially for new users ("cold-start" problem).
Existing solutions often struggle to balance privacy with the need for personalization and struggle to account for the nuanced differences between individuals.
Methods
DivAR proposes a novel approach combining Federated Learning and Meta-Learning with an Attention Mechanism to address these challenges:
- Federated Learning: This allows models to be trained locally on user devices without sharing raw data. Only model updates are sent to a central server, preserving privacy.
- Meta-Learning (specifically, a Reptile-like approach): This enables the model to "learn to learn." A central meta-model learns general feature representations that can be quickly adapted to new tasks or users.
- Individual Clustering: Before meta-learning, individuals are grouped into clusters based on their social factors (e.g., personality, mental state) and behavioral patterns. This allows for more targeted learning within each group.
- Attention Mechanism: Within each decentralized client model, a CNN-based attention module is employed. This module learns to selectively focus on the most relevant sensor features for a specific cluster, effectively extracting individual-specific features from the globally learned representations.
The architecture involves a central server that maintains a shared embedding network and decentralized client models that learn cluster-specific features using the attention mechanism.
Datasets
The authors created two new datasets for their experiments:
- Dataset 1 & Dataset 2: These datasets consist of sensor readings from naturally used smartphones, annotated with activity information. They were collected from multiple individuals in real-life environments. Crucially, these datasets also include social characteristics (personality, mental health, behavior patterns) collected via questionnaires, which are used for individual clustering. These datasets are described as "multi-individual heterogeneous."
Key Insights and Ideas
- Clustering is Key: Grouping individuals based on their characteristics significantly improves the model's ability to handle heterogeneity and generalize across users. The paper shows that DivAR-cluster (with clustering) consistently outperforms DivAR-individual (without clustering).
- Attention for Personalization: The attention mechanism is vital for extracting individual-specific features from shared representations, allowing the model to adapt to diverse behavioral patterns.
- Beyond Basic Features: Incorporating social and behavioral factors for clustering is more effective than relying solely on basic demographic information or even just behavioral type.
- Federated Meta-Learning Synergy: The combination of federated learning for privacy and meta-learning for rapid adaptation is a powerful paradigm for personalized HAR in real-world, distributed settings.
Key Results
- Superior Performance: DivAR consistently outperformed baseline methods (Random Forest, DeepSense, AttenSense, Meta-HAR) in terms of accuracy and macro F1-score across three datasets (two custom, one mixed).
- Significant Gains: DivAR-cluster achieved performance gains of up to 5.50% in accuracy and 9.61% in macro F1-score compared to its individual-focused counterpart.
- Effective Cold-Start Adaptation: DivAR demonstrated strong performance in adapting to new users, outperforming Meta-HAR by effectively capturing feature-level discrepancies.
- Computational Efficiency: DivAR generally requires fewer training epochs to achieve its best performance compared to other models.
Limitations and Future Work
- Computational Resources: The paper acknowledges that deploying deep learning models on edge devices like smartphones requires significant computational resources, and models may need compression for smaller devices.
- Unsupervised Clustering: The authors suggest that more unsupervised methods for clustering users could be explored to reduce reliance on collecting explicit user information.
- Future Work: The authors plan to explore additional social factors for clustering and conduct correlation analysis to enhance model explainability.
Summary updated